Probabilistic inference using weighted-integrals-and-sums-by-hashing for object tracking

ABSTRACT

Systems, methods, and devices for sensor fusion are disclosed herein. A system for sensor fusion includes one or more sensors, a model component, and an inference component. The model component is configured to calculate values in a joint-probabilistic graphical model based on the sensor data. The graphical model includes nodes corresponding to random variables and edges indicating correlations between the nodes. The inference component is configured to detect and track obstacles near a vehicle based on the sensor data and the model using a weighted-integrals-and-sums-by-hashing (WISH) algorithm.

TECHNICAL FIELD

The disclosure relates generally to methods, systems, and apparatusesfor automated driving or for assisting a driver, and more particularlyrelates to methods, systems, and apparatuses for object detection andtracking.

BACKGROUND

Autonomous vehicles and driving assistance systems are currently beingdeveloped and deployed to provide safety, reduce an amount of user inputrequired, or even eliminate user involvement entirely during driving ofa vehicle. For example, some driving assistance systems, such as crashavoidance systems, may monitor driving, positions, and velocities of thevehicle and other objects while a human is driving. When the systemdetects that a crash or impact is imminent the crash avoidance systemmay intervene and apply a brake, steer the vehicle, or perform otheravoidance or safety maneuvers. As another example, autonomous vehiclesmay drive and navigate a vehicle with little or no user input. However,due to the dangers involved in driving and the costs of vehicles, it isextremely important that autonomous vehicles and driving assistancesystems operate safely and are able to accurately navigate roads,observe their surroundings and avoid objects.

BRIEF DESCRIPTION OF THE DRAWINGS

Non-limiting and non-exhaustive implementations of the presentdisclosure are described with reference to the following figures,wherein like reference numerals refer to like parts throughout thevarious views unless otherwise specified. Advantages of the presentdisclosure will become better understood with regard to the followingdescription and accompanying drawings where:

FIG. 1 is a schematic block diagram illustrating an implementation of avehicle control system that includes an automated driving/assistancesystem;

FIG. 2 is a schematic diagram illustrating a top view of an example roadenvironment;

FIG. 3 is a schematic block diagram illustrating a sensor fusionalgorithm, according to one implementation;

FIG. 4 is a schematic block diagram illustrating sensor fusion,according to one implementation;

FIG. 5 is a schematic block diagram illustrating sub-components of asensor fusion component, according to one implementation;

FIG. 6 is a schematic block diagram illustrating a method for sensorfusion, according to one implementation; and

FIG. 7 is a schematic block diagram illustrating a method for sensorfusion, according to another implementation.

DETAILED DESCRIPTION

Availability of multi-modal sensors (cameras, radars, ultrasonics) on avehicle, each with their unique strengths and weaknesses, and withcomplimentary properties, allows a system to leverage the sensors in asynergistic way to perform accurate and robust object detection andtracking. Utilizing multiple sensors in a way that enhances theirstrengths, rather than their weaknesses, requires high-fidelity modelsand advanced algorithmic techniques. Off-the-shelf probabilisticinference and modeling technology, like Kalman filters, are restrictedto Gaussian, unimodal dynamics and use Markov chain models to representthe probabilistic system, which suffers from having too sparse samplingand low guarantees on safety.

Furthermore, sensor fusion algorithms (like Kalman filtering, MarkovChain Monte-Carlo (MCMC) sampling etc.) are either not scalable or notaccurate enough for object detection and tracking MCMC techniques arestill the most widely used and are the workhorse of statisticalinference. Unfortunately, such methods typically do not provide tightguarantees on the accuracy of the results. At least some approachesdisclosed herein introduce a fundamentally new paradigm for statisticalinference, which is a very different and promising alternative to theseexisting techniques and yields provably accurate results in a range ofproblem domains.

The present disclosure proposes application of an inference anddecision-making technique based on dimensionality reductions to sensorfusion problems. One recently developed example is theWeighted-Integrals-And-Sums-By-Hashing technique or algorithm (WISH)disclosed in “Taming the Curse of Dimensionality: Discrete Integrationby Hashing and Optimization” by Stefano Ermon, Carla Gomes, AshishSabharwal, and Bart Selman, presented in the Proceedings of theInternational Machine Learning Society, 2013, which is herebyincorporated herein in its entirety by this reference. In oneembodiment, the use of the WISH algorithm or other dimensionalityreduction algorithms work well when the underlying statistical modelscombine probabilistic information with causal or deterministicconstraints arising from domain knowledge (such as physical laws).

In one embodiment, methods of the present disclosure may be understoodas including two or more levels. The first level may include creation ofa joint-probabilistic graphical model that captures the uncertainty andrelationship between different sensors. In one embodiment, thejoint-probabilistic graphical model is created using rigoroustheoretical analysis with a principled experimental component, drawingupon ideas from probabilistic reasoning, constraint optimization andmachine learning. In one embodiment, the joint-probabilistic graphicalmodel includes continuous parameters (such as for speed, position,velocity, or the like) as well as discrete parameters (such as sensorfailure status, occupancy of a region by an object, or the like) modeledwithin it. These parameters, in one implementation, enable the estimateof an object's position and velocity, a metric for determining theconfidence level of finding the position and velocity, and/or discretedetection/tracking of a sensor's failure (confidence level). Based oninformation obtained from the model, an inference technique like theWISH algorithm may then be used to perceive and understand the scenearound the vehicle. The result of the WISH algorithm may includeconfidence bounds for different inferences and hence provideprobabilistic safety to the system. It will be appreciated that the WISHalgorithm is a better approach for object detection than existing MonteCarlo estimation techniques in terms of accuracy, robustness andconvergence rates and is highly scalable and parallelizable.

In one embodiment, in order to generate a graphical model for a specificsensor/vehicle system, sensor data is collected from a variety ofsensors positioned on a vehicle and driven through multiple real-lifescenarios or scenarios created in a virtual driving environment. Thedata gathered from real-world and virtual-world scenarios may be usedfor developing the joint-probabilistic graphical model. This model mayinclude learning parameters developed through deep learning concepts aswell as incorporation of physical system knowledge. The WISH algorithmmay then be used for refining or defining parameters. The WISH algorithmmay then be used for inference and decision making in the scene fordifferent discrete and continuously tracked variables, such as positionand speed as well as the discrete components, such as sensor failure.Virtual data may be obtained via Simulink® models (or other models) thathave been developed and used to test the functioning of the algorithm.The accuracy of the algorithm is determined by comparison with groundtruth obtained from the virtual environment. After virtual testing, thealgorithm may then be tested on real-world data collected for variousscenarios. An exploration of different levels of fusion of sensor datais performed to analyze the best level at which information fusionshould take place for different applications.

As discussed above, one embodiment of a sensor fusion approach mayinclude a two-layered system with one part focusing on creating a singlejoint-probabilistic model of the dynamics of a system (such as of avehicle and sensors) and a second part that may be used to inferdifferent insights based on the model. In one embodiment, thejoint-probability graphical model may be hard-coded or may be learnedand modified using artificial intelligence or machine learning.

One or more approaches for performing sensor fusion presented herein aredifferent than the state-space approach for other methods that have asingle model performing the fusion. The one or more approaches forperforming sensor fusion differ from the state-space filtering, becausefusion takes place at both levels, namely at the probabilistic modellevel and also at the inference level above it. Additionally, unlikeexisting systems, one or more approaches provided herein provide aquantitative confidence measure of safety of the inference. Compared toexisting techniques, embodiments of the present disclosure are highlyscalable, parallelizable, and converge significantly faster, which areall advantageous for active safety. Thus the present disclosure enablesaccurate detection and tracking of obstacles in an environment by fusinginformation from multi-modal sensors to aid in development ofdriver-assist features and capabilities for autonomous driving.

In the following disclosure, reference is made to the accompanyingdrawings, which form a part hereof, and in which is shown by way ofillustration specific implementations in which the disclosure may bepracticed. It is understood that other implementations may be utilizedand structural changes may be made without departing from the scope ofthe present disclosure. References in the specification to “oneembodiment,” “an embodiment,” “an example embodiment,” etc., indicatethat the embodiment described may include a particular feature,structure, or characteristic, but every embodiment may not necessarilyinclude the particular feature, structure, or characteristic. Moreover,such phrases are not necessarily referring to the same embodiment.Further, when a particular feature, structure, or characteristic isdescribed in connection with an embodiment, it is submitted that it iswithin the knowledge of one skilled in the art to affect such feature,structure, or characteristic in connection with other embodimentswhether or not explicitly described.

Implementations of the systems, devices, and methods disclosed hereinmay comprise or utilize a special purpose or general-purpose computerincluding computer hardware, such as, for example, one or moreprocessors and system memory, as discussed in greater detail below.Implementations within the scope of the present disclosure may alsoinclude physical and other computer-readable media for carrying orstoring computer-executable instructions and/or data structures. Suchcomputer-readable media can be any available media that can be accessedby a general purpose or special purpose computer system.Computer-readable media that store computer-executable instructions arecomputer storage media (devices). Computer-readable media that carrycomputer-executable instructions are transmission media. Thus, by way ofexample, and not limitation, implementations of the disclosure cancomprise at least two distinctly different kinds of computer-readablemedia: computer storage media (devices) and transmission media.

Computer storage media (devices) includes RAM, ROM, EEPROM, CD-ROM,solid state drives (“SSDs”) (e.g., based on RAM), Flash memory,phase-change memory (“PCM”), other types of memory, other optical diskstorage, magnetic disk storage or other magnetic storage devices, or anyother medium which can be used to store desired program code means inthe form of computer-executable instructions or data structures andwhich can be accessed by a general purpose or special purpose computer.

An implementation of the devices, systems, and methods disclosed hereinmay communicate over a computer network. A “network” is defined as oneor more data links that enable the transport of electronic data betweencomputer systems and/or modules and/or other electronic devices. Wheninformation is transferred or provided over a network or anothercommunications connection (either hardwired, wireless, or a combinationof hardwired or wireless) to a computer, the computer properly views theconnection as a transmission medium. Transmissions media can include anetwork and/or data links which can be used to carry desired programcode means in the form of computer-executable instructions or datastructures and which can be accessed by a general purpose or specialpurpose computer. Combinations of the above should also be includedwithin the scope of computer-readable media.

Computer-executable instructions comprise, for example, instructions anddata which, when executed at a processor, cause a general purposecomputer, special purpose computer, or special purpose processing deviceto perform a certain function or group of functions. The computerexecutable instructions may be, for example, binaries, intermediateformat instructions such as assembly language, or even source code.Although the subject matter has been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the subject matter defined in the appended claims is notnecessarily limited to the described features or acts described above.Rather, the described features and acts are disclosed as example formsof implementing the claims.

Those skilled in the art will appreciate that the disclosure may bepracticed in network computing environments with many types of computersystem configurations, including, in-dash computers, personal computers,desktop computers, laptop computers, message processors, hand-helddevices, multi-processor systems, microprocessor-based or programmableconsumer electronics, network PCs, minicomputers, mainframe computers,mobile telephones, PDAs, tablets, pagers, routers, switches, variousstorage devices, and the like. The disclosure may also be practiced indistributed system environments where local and remote computer systems,which are linked (either by hardwired data links, wireless data links,or by a combination of hardwired and wireless data links) through anetwork, both perform tasks. In a distributed system environment,program modules may be located in both local and remote memory storagedevices.

Further, where appropriate, functions described herein can be performedin one or more of: hardware, software, firmware, digital components, oranalog components. For example, one or more application specificintegrated circuits (ASICs) can be programmed to carry out one or moreof the systems and procedures described herein. Certain terms are usedthroughout the following description and Claims to refer to particularsystem components. As one skilled in the art will appreciate, componentsmay be referred to by different names. This document does not intend todistinguish between components that differ in name, but not function.

Referring now to the figures, FIG. 1 illustrates a vehicle controlsystem 100 that includes an automated driving/assistance system 102. Theautomated driving/assistance system 102 may be used to automate orcontrol operation of a vehicle or to provide assistance to a humandriver. For example, the automated driving/assistance system 102 maycontrol one or more of braking, steering, acceleration, lights, alerts,driver notifications, radio, or any other auxiliary systems of thevehicle. In another example, the automated driving/assistance system 102may not be able to provide any control of the driving (e.g., steering,acceleration, or braking), but may provide notifications and alerts toassist a human driver in driving safely. The automateddriving/assistance system 102 includes a sensor fusion component 104,which may detect or track objects based on a graphical model of avehicle corresponding to the control system 100 and data gathered by oneor more sensors. For example, the sensor fusion component 104 may infera location, speed, velocity, object type, or any other details aboutphysical objects near the vehicle. Additionally, the sensor fusioncomponent 104 may also determine a confidence level in one or moredetails about physical objects or obstacles.

The vehicle control system 100 also includes one or more sensorsystems/devices for detecting a presence of nearby objects ordetermining a location of a parent vehicle (e.g., a vehicle thatincludes the vehicle control system 100). For example, the vehiclecontrol system 100 may include one or more radar systems 106, one ormore LIDAR systems 108, one or more camera systems 110, a globalpositioning system (GPS) 112, and/or one or more ultrasound systems 114.The one or more sensor systems/devices may include any other sensors,such as wheel encoders to detect the speed of the vehicle and/or thedistance rotated by the wheels of the vehicle or other sensors to detectother objects or detect a location or movement of the vehicle.

The vehicle control system 100 may include a data store 116 for storingrelevant or useful data for navigation and safety such as a drivinghistory, map data, or other data. In one embodiment, the data store 116may store a joint-probabilistic graphical model that models the system100, including any sensors of the system 100. The vehicle control system100 may also include a transceiver 118 for wireless communication with amobile or wireless network, other vehicles, infrastructure, or any othercommunication system. The vehicle control system 100 may include vehiclecontrol actuators 120 to control various aspects of the driving of thevehicle, such as electric motors, switches or other actuators to controlbraking, acceleration, steering or the like. The vehicle control system100 may also include one or more displays 122, speakers 124, or otherdevices so that notifications to a human driver or passenger may beprovided. The display 122 may include a heads-up display, a dashboarddisplay or indicator, a display screen, or any other visual indicator,which may be seen by a driver or passenger of a vehicle. The speakers124 may include one or more speakers of a sound system of a vehicle ormay include a speaker dedicated to driver notification.

It will be appreciated that the embodiment of FIG. 1 is given by way ofexample only. Other embodiments may include fewer or additionalcomponents without departing from the scope of the disclosure.Additionally, illustrated components may be combined or included withinother components without limitation. For example, the sensor fusioncomponent 104 may be separate from the automated driving/assistancesystem 102 and the data store 116 may be included as part of theautomated driving/assistance system 102 and/or part of the sensor fusioncomponent 104.

The radar system 106 may include any radar system well known in the art.Radar system 106 operation and performance is generally well understood.In general, a radar system 106 operates by transmitting radio signalsand detecting reflections off objects. In ground applications, the radarmay be used to detect physical objects, such as other vehicles, parkingbarriers or parking chocks, landscapes (such as trees, cliffs, rocks,hills, or the like), road edges, signs, buildings, or other objects. Theradar system 106 may use the reflected radio waves to determine a size,shape, distance, surface texture, or other information about a physicalobject or material. For example, the radar system 106 may sweep an areato obtain data about objects within a specific range and viewing angleof the radar system 106. In one embodiment, the radar system 106 isconfigured to generate perception information from a region near thevehicle, such as one or more regions nearby or surrounding the vehicle.For example, the radar system 106 may obtain data about regions of theground or vertical area immediately neighboring or near the vehicle. Theradar system 106 may include one of many commercially available radarsystems. In one embodiment, the radar system 106 may provide perceptiondata including a two-dimensional or three-dimensional map or model tothe automated driving/assistance system 102 for reference or processing.

The LIDAR system 108 may include any LIDAR system known in the art.Principles of operation and performance of LIDAR systems are generallywell understood. In general, the LIDAR system 108 operates by emittingvisible wavelength or infrared wavelength lasers and detectingreflections of the laser light off objects. In ground applications, thelasers may be used to detect physical objects, such as other vehicles,parking barriers or parking chocks, landscapes (such as trees, cliffs,rocks, hills, or the like), road edges, signs, buildings, or otherobjects. The LIDAR system 108 may use the reflected laser light todetermine a size, shape, distance, surface texture, or other informationabout a physical object or material. For example, the LIDAR system 108may sweep an area to obtain data or objects within a specific range andviewing angle of the LIDAR system 108. For example, the LIDAR system 108may obtain data about regions of the ground or vertical area immediatelyneighboring or near the vehicle. The LIDAR system 108 may include one ofmany commercially available LIDAR systems. In one embodiment, the LIDARsystem 108 may provide perception data including a two-dimensional orthree-dimensional model or map of detected objects or surfaces.

The camera system 110 may include one or more cameras, such as visiblewavelength cameras or infrared cameras. The camera system 110 mayprovide a video feed or periodic images, which can be processed forobject detection, road identification and positioning, or otherdetection or positioning. In one embodiment, the camera system 110 mayinclude two or more cameras, which may be used to provide ranging (e.g.,detect a distance) for objects within view.

The GPS system 112 is one embodiment of a positioning system that mayprovide a geographical location of the vehicle based on satellite orradio tower signals. GPS systems 112 are well known and widely availablein the art. Although GPS systems 112 can provide very accuratepositioning information, GPS systems 112 generally provide little or noinformation about distances between the vehicle and other objects.Rather, they simply provide a location, which can then be compared withother data, such as maps, to determine distances to other objects,roads, or locations of interest.

The ultrasound system 114 may be used to detect objects or distancesbetween a vehicle and objects using ultrasound waves. For example, theultrasound system 114 may emit ultrasonic waves from a location on ornear a bumper or side panel location of a vehicle. The ultrasonic waves,which can travel short distances through air, may reflect off otherobjects and be detected by the ultrasound system 114. Based on an amountof time between emission and reception of reflected ultrasonic waves,the ultrasound system 114 may be able to detect accurate distancesbetween a parent vehicle and other vehicles or any other objects. Due toits shorter range, the ultrasound system 114 may be more useful todetect objects during parking, detect imminent collisions duringdriving, or detect nearby vehicles or objects during parking or driving.

In one embodiment, the data from radar system(s) 106, LIDAR system(s)108, camera system(s) 110, and ultrasound system(s) 114 may be processedby the sensor fusion component 104 to quickly and accurately obtaininformation about nearby objects, such as their position, speed relativeto the vehicle, direction of travel, shape, or the like. In oneembodiment, the sensor fusion component 104 obtains more accurateinformation about an object than any single sensor system (or any singlesensor type) can obtain independently.

The data store 116 stores map data, a driving history, a probabilisticgraphical model, and/or other data, which may include other navigationaldata, settings, or operating instructions for the automateddriving/assistance system 102. The map data may include location data,such as GPS location data, for roads, parking lots, parking stalls, orother places where a vehicle may be driven or parked. For example, thelocation data for roads may include location data for specific lanes,such as lane direction, merging lanes, highway or freeway lanes, exitlanes, or any other lane or division of a road. The location data mayalso include locations for each parking stall in a parking lot or forparking stalls along a road. In one embodiment, the map data includeslocation data about one or more structures or objects on or near theroads or parking locations. For example, the map data may include dataregarding GPS sign location, bridge location, building or otherstructure location, or the like. In one embodiment, the map data mayinclude precise location data with accuracy within a few meters orwithin sub-meter accuracy. The map data may also include location datafor paths, dirt roads, or other roads or paths, which may be driven by aland vehicle.

The driving history (or drive history) may include location data and/orsensor data for past trips or parking locations of the vehicle. Forexample, the driving history may include GPS location data for theprevious trips or paths taken. As another example, the driving historymay include distance or relative location data with respect to lanelines, signs, road border lines, or other objects or features on or nearthe roads. The distance or relative location data may be determinedbased on GPS data, radar data, LIDAR data, camera data, or other sensordata gathered during the previous or past trips taken by the vehicle. Asyet another example, the driving history may include detailed sensordata for events experienced by the vehicle, including resulting events.In one embodiment, the automated driving/assistance system 102 isconfigured to log driving data to the data store 116 for and during anytrips or drives taken by the vehicle.

The transceiver 118 is configured to receive signals from one or moreother data or signal sources. The transceiver 118 may include one ormore radios configured to communicate according to a variety ofcommunication standards and/or using a variety of different frequencies.For example, the transceiver 118 may receive signals from othervehicles. Receiving signals from another vehicle may also be referencedherein as vehicle-to-vehicle (V2V) communication. In one embodiment, thetransceiver 118 may also be used to transmit information to othervehicles to potentially assist them in locating vehicles or objects.During V2V communication the transceiver 118 may receive informationfrom other vehicles about their locations, other traffic, accidents,road conditions, the locations of parking barriers or parking chocks, orany other details that may assist the vehicle and/or automateddriving/assistance system 102 in driving accurately or safely.

The transceiver 118 may receive signals from other signal sources thatare at fixed locations. In one embodiment, receiving or sending locationdata from devices or towers at fixed locations is referenced herein asvehicle-to-infrastructure (V2X) communication. In one embodiment, theterm V2X communication may also encompass V2V communication.

In one embodiment, the transceiver 118 may send and receive locationdata, sensor data, decision making data, or any other data via a mobilenetwork or cell connection. For example, the transceiver 118 may receiveupdated data for a probabilistic graphical model, or other configurationdata for sensors. In one embodiment, the transceiver 118 may send andreceive data regarding the detection and tracking of objects for storageon a remote server, such as part of a cloud service for improving sensorfusion algorithms.

In one embodiment, the automated driving/assistance system 102 isconfigured to control driving or navigation of a parent vehicle. Forexample, the automated driving/assistance system 102 may control thevehicle control actuators 120 to drive a path on a road, a parking lot,a driveway or other location. For example, the automateddriving/assistance system 102 may determine a path and a speed to drivebased on information or perception data provided by any of thecomponents 106-118. In one embodiment, the sensor fusion component 104is configured to make inferences or decisions regarding details ofobstacles. For example, the sensor fusion component 104 may providelocations, velocities, or other information about objects to anavigation or driving system so that the objects can be avoided.

FIG. 2 is a schematic top view of a road 200 with a vehicle 202traveling on the road 200. The vehicle 202 may include the system 100 ofFIG. 1. In one embodiment, one or more sensors, such as a camera system110 of the vehicle may have a viewing area 204 indicated by arcuate,dotted lines. The viewing area is illustrative only and may extend inany direction or all directions around the vehicle 202. Furthermore, theregion covered by a sensor may be larger or smaller based on sensortype, orientation, or the like. The vehicle 202, or a sensor fusioncomponent 104 of the vehicle 202, may receive sensor data that includesinformation about objects, surfaces, or the like within the viewing area204 and make inferences about object location, velocity, or the like. Inone embodiment, the presence of certain objects or environmentalattributes may be determined by the vehicle 202 based on the datagathered within the viewing area 204. In one embodiment, the vehicle 202may process sensor data to detect and track another vehicle 208, signs210 or 212, an intersecting road 214, debris in the roads, animals,people, or other objects. In one embodiment, the viewing area 204 may belogically subdivided into a plurality of regions or cells to create alogical grid. Detection of objects may be performed by detecting objectswithin a specific cell. The vehicle 202 may then know whether or not toavoid a cell based on whether an object or obstacle was detected there.

In addition to perception data, the vehicle 202 may obtain informationfrom a stored map, stored driving history, or from wireless signals. Forexample, an infrastructure transmitter 206 is shown near the road 200,which may provide specific positioning, environmental attribute details,or other information to the vehicle. As further examples, the vehicle202 may receive information from other vehicles, such as vehicle 208, orfrom a wireless communication network. Based on the informationreceived, the vehicle 202 may determine a location, velocity, identity,or the like about one or more detected objects or may update analgorithm or joint-probabilistic graphical model.

FIG. 3 is a schematic block diagram illustrating data flow and featuresof a sensor fusion algorithm. Sensor data from radar, camera,ultrasound, GPS, and/or other sensors may be received for sensor fusion.A joint-probabilistic graphical model includes nodes, edges orconnections reflecting dependent (or independent) relationships, andtables or values that reflect probabilities at each node or forrelationships between nodes. For example, one or more nodes mayrepresent a measured sensor value for specific sensors and one or moreother nodes may represent binary occupancy for a specific region.

The values and organization for the relationships (edges) and nodes inthe joint-probabilistic graphical model may reflect continuous anddiscrete components or aspects of a vehicle or a sensor system. Forexample, continuous components or parameters may include those that cantake on a continuous number of different values such as speed, distance,location, x-coordinate, y-coordinate, z-coordinate, velocity, or thelike. Examples of discrete components or parameters include those thatcan take on two or more specific values (e.g., predefined states such astrue/false, or one of a plurality of identification categories) such assensor failure status, object identification (e.g., vehicle, animal,pedestrian, cyclist, curb, tree, debris, etc.), occupancy of a cell in alogical grid (present or not present), or the like.

The values and organization for the relationships and nodes in thejoint-probabilistic graphical model may include values determined orupdated using machine learning. For example, during creation of themodel the values and/or relationships between nodes may be created ormodified based on relationships or values detected during machinelearning. Machine learning may be used to process large amounts ofactual sensor data with respect to known (or accurate) values for one ormore relationships that a vehicle will detect or track later. In oneembodiment, a unidirectional graphical model may be generated withfunctions having unknown or estimated parameters describing correlationsor relationships between nodes. Using machine learning, values for theunknown or estimated parameters may be determined or refined. Forexample, after the structure of the graphical model is determined, theWISH algorithm may be used to iterate over real-world or virtual-worldsensor data to generate values for unknown or estimated parameters.

Furthermore, after the occurrence of driving events has taken place,when greater detail or accuracy is obtained, the comparison of accuracyof any inferences obtained through sensor fusion may be compared to thegreater detail or accuracy obtained later (e.g., after an event hasended and more accurate results are obtained). Modifications to thejoint-probabilistic model may be made to improve the accuracy of themodel with respect to the driving events.

The values and organization for the relationships and nodes in thejoint-probabilistic graphical model may include values reflecting thedynamics and uncertainty for a vehicle, a sensor system, and/or aspecific sensor. For example, the relationships may indicate how a valuemeasured by one sensor for a specific object relates to a value measuredby a different sensor (e.g., a different sensor type) for the samespecific object. Furthermore, the values may reflect how measurements bysensors change based on temperature, lighting levels, weatherconditions, or the like. Uncertainty in a dynamical system can occur dueto two reasons: the first is noise in the sensor measurementsrepresenting the state of the system; and the second is noise in thestate of the system itself (e.g., the system may not move according tothe model defining its motion (for example, a vehicle traveling in astraight line)).

The values and organization for the relationships and nodes in thejoint-probabilistic graphical model may reflect laws of physics, whichbound operation of a vehicle or a sensor system. For example, physicallimitations on the movements of vehicles (or other objects) as well asthe physical limitations on sensors may also be reflected by values,relationships, and nodes in the joint-probabilistic graphical model. Forexample, functions for edges that describe relationships between nodesmay be generated to eliminate results that are not possible in the realworld based on physical laws and limitations of vehicle movement and/orthe sensors.

The sensor data may then be processed based on the joint-probabilisticgraphical model using an inference and decision making algorithm, suchas the WISH algorithm. In one embodiment, the WISH algorithm may be usedto detect the presence of an object within one or more cells of alogical grid of an area near a vehicle. Systems of many random variablesmay take on an exponential number of possible joint configurations(random variable settings). This presents a problem because inferenceoften requires computation of a partition function, which is a weightedsum over all (which may be exponentially many) configurations. The WISHalgorithm, as used herein, approximates (with rigorous approximationbounds) this intractable counting problem using a small number ofcomparatively easier optimization problems. Moreover, the WISH algorithmcan be parallelized for efficient computation during driving oroperation of a vehicle. Model selection further compounds thedimensionality problem of inference, primarily because evaluating aparticular parameter setting entails computing the partition functionanew. The application of the WISH algorithm as presented herein enablestractable and automatic generation of information fusion models fornovel multi-sensor platforms, which may be especially useful when onehas limited domain knowledge over how to weight the information fromdisparate sensors in a given context.

FIG. 4 is a schematic block diagram illustrating one embodiment ofsensor fusion using a WISH algorithm. Radar, LIDAR, camera, GPS, andother systems may provide information via respective signal interfaces.Sensor fusion takes place with the information from the sensors ormeasurement systems. The sensor fusion may use a Kalman filter, aparticle filter, a WISH algorithm, and/or deep learning algorithms toproduce inferred values for object detection, speed limits, drivingbehavior decisions, probability or confidence values, or other valuesthat may be helpful for an automated driving/assistance system 102.Based on the sensor fusion output, and/or any known information about anevent, the accuracy of the inferred values may be validated and/or thesensor fusion algorithm may be updated. For example, a probabilisticgraphical model may be updated based on the additional data gatheredduring a real or virtual driving event.

A graphical model may be used for modeling a sensor system. In oneembodiment, the probabilistic graphical model for this approach wouldpossibly be an undirected graphical model. The nodes of the graphicalmodel may include collections of random variables. In one embodiment,the nodes would capture binary random variables like occupancy (e.g., ofa cell of a logical grid representing a region near a sensor or avehicle), categorical variables like material type, continuous variableslike reflectivity and spectral signature, or the like. The edges maydescribe how pairs of random variables correlate with each other. Forexample, on every edge in the graph a compatibility function maydescribe how the nodes are connected. Parameters developed via machinelearning may help describe these functions, which can capture physics ofthe different sensing modalities (e.g., of LIDAR, radar, camera, or thelike).

In one embodiment, nodes in the system may represent binary variablesdescribing occupancy at different points or regions. For example, anarea near a vehicle may be logically divided up into a grid with a noderepresenting occupancy within a specific region or cell of the grid. Theedges may include corresponding joint probability distribution functions(PDF) that describe a mathematical relationship between two nodes. Thejoint PDFs may include parameters (like noise levels for differentsensor measurements) that are learned. Another example could be learningthe characteristic scale of different objects in the environment overtime using parameters that describe the material properties of theobjects etc.

The WISH algorithm could be used during preparation and design of agraphical model (e.g., during an offline phase) and/or could be usedduring active operations of a vehicle for object tracking and decisionmaking (e.g., during an online phase). As used herein, the term “online”refers to a sensor system that is actively assisting driving of avehicle; while the term “offline” refers to a sensor system that is notactively assisting driving of a vehicle. For example, before a graphicalmodel is finalized it may be used as a model for machine learning,testing, or the like during an offline phase. Once the graphical modelis installed in a vehicle it may be used by the vehicle during activedriving scenarios during an online phase.

In the offline “learning” phase, the WISH algorithm may be provided withpre-collected sensor data or modeled sensor data and the structure ofthe graphical model and its parameterization, but not the parametervalues. The WISH algorithm would help learn the parameters of thegraphical model in an iterative fashion going over the pre-collectedvirtual or real data. Once the probabilistic graphical model parametersare learned, the WISH algorithm would be provided with the data fromvarious sensors during active driving or operation of a vehicle (e.g.,during an online phase). The WISH algorithm may then use the data fromthe sensors in conjunction with the structure and learned parameters ofthe graphical model. The WISH algorithm would process this data and thenprovide an inference output that includes a probability of occupancy ofa certain grid cell, its velocity, or other information about a gridcell or object in the grid cell. This information would then be used fordetecting and tracking objects around the vehicle during driving.

FIG. 5 is a schematic block diagram illustrating sub-components of asensor fusion component 104, according to one embodiment. The sensorfusion component 104 includes a sensor data component 502, a modelcomponent 504, an inference component 506, a notification component 508,and a learning component 510. The components 502-510 are given by way ofillustration only and may not all be included in all embodiments. Infact, some embodiments may include only one or any combination of two ormore of the components 502-510. Some of the components 502-510 may belocated outside the sensor fusion component 104, such as within theautomated driving/assistance system 102 or elsewhere.

The sensor data component 502 is configured to receive sensor data fromone or more sensors. In one embodiment, the sensor data may includeperception data from one or more perception sensors for observing aregion near a vehicle. The perception data may include data produced byone or more of a camera system 110, a radar system 106, a LIDAR system108, an ultrasound system 114, or any other system. In one embodiment,the perception data may include sensor measurements for one or moreregions near the vehicle. The sensor data may also include informationfrom one or more sensors on the vehicle including information from a GPSsystem 112, wheel encoders, or any other systems that gather data abouta location or movement of a vehicle.

The model component 504 models a sensing system of a vehicle, forexample, a vehicle in which the sensor fusion component 104 or automateddriving/assistance system 102 is located. In one embodiment, the modelcomponent 504 may store or calculate a graphical model that modelsoperations, relationships, and/or values for a plurality of sensors ofthe vehicle. The graphical model may include nodes that correspond toone or more continuous variables for an object detected by two or moresensors, such as reflectivity and spectral signature of an obstacle. Thegraphical model may include nodes that correspond to one or morecategorical variables for an obstacle, such as an object type and amaterial type. The graphical model may include nodes that correspond toone or more binary variables, such as occupancy of a cell by an object.The graphical model may include one or more edges or connections betweennodes that indicate a relationship between the variables represented bythe nodes. In one embodiment, the model may include a function forcalculating a value at a node based on the sensor data. The function mayinclude one or more parameters that reflect machine learning or measuredrelationships between nodes or between sensor data and/or calculationrelationships between sensor measurements and modeled variables. In oneembodiment, one or more parameters may be determined via machinelearning based on real-world or virtual data and known ground truth froma real or virtual environment. In one embodiment, the model includesjoint probabilistic functions for one or more edges describing how nodescorresponding to the edges are mathematically related. The parameters ofthe model may be machine learned or may be calculated based ondeterministic constraints based on physical laws for the vehicle and/orprobabilistic values for confidence levels for detecting and trackingthe physical objects based on uncertainty in sensor measurements.

In one embodiment, a region near or surrounding a vehicle may belogically divided into a two-dimensional or three-dimensional gridhaving a plurality of cells. The grid may include a square, circular,oval, spherical, or any other type of grid. In one embodiment, each cellof the grid may have a corresponding node in a graphical model, whichmodels for an occupancy (e.g., presence of an object in that cell),velocity, material type, object recognition, or any other objectdetection and tracking information. For example, each cell of the gridmay correspond to a specific region near a vehicle (e.g., with a fixedlocation with respect to a moving vehicle) and one or more aspects of anobject (or absence of an object) may be inferred for that cell based onthe model and the most recently gathered sensor data.

In one embodiment, the graphical model may be stored in computerreadable media, such as in the data store 116 of FIG. 1. In oneembodiment, the model component 504 may retrieve the graphical modelfrom memory. In one embodiment, the model component 504 may calculatevalues for each node of the graphical model based on recent sensor dataand the functions in the model. For example, the graphical model mayinclude variables for which sensor measurements may be substituted tocalculate a value for each specific node of the graphical model. Themodel component 504 may provide the model with calculated values to theinference component 506 for inferring information about the position,speed, identity, or other aspects of a detected object. In oneembodiment, the model component 504 may provide the model and the sensordata to the inference component 506.

The inference component 506 is configured to infer information about thepresence or absence of obstacles and/or properties of detectedobstacles. In one embodiment, the inference component 506 is configuredto process the sensor data and the model using a dimensionalityreduction algorithm to detect and track physical objects. For example,the inference component 506 may determine whether an object (such as avehicle, person, animal, curb, tree, or any other obstacle) is presentat a specific location with relation to a vehicle. In one embodiment,the inference component 506 may determine an identity of the object, aspeed or velocity of the object, a predicted behavior of the object, orthe like. In one embodiment, the inference component 506 may determine aprobability of accuracy for a specific inference.

In one embodiment, the inference component 506 may use the WISHalgorithm to process the sensor data and a graphical model to determineone or more inferences. For example, the inference component 506 mayperform hashes and sums on the model and/or sensor values to determinewhether an obstacle is present and other information about the vehicle.

The notification component 508 is configured to provide one or moreinferences determined by the inference component 506 to an automateddriving/assistance system 102. For example, the notification component508 may provide cell values of a grid corresponding to a regionsurrounding a vehicle to the automated driving/assistance system 102.For example, each cell may include an indication of occupancy, avelocity of a detected object, an object classification, and/or aprobability value for probability, velocity, or classification. In oneembodiment, the automated driving/assistance system 102 may makedecisions based on the inferences and probabilities provided by thesensor fusion component 104. In one embodiment, the notificationcomponent 508 may provide a location or a velocity of the physicalobjects to an automated driving/assistance system 102. In oneembodiment, the automated driving/assistance system 102 may select adriving path based on the detection and tracking of the physicalobjects.

The learning component 510 is configured to perform machine learning toupdate operation of the sensor fusion component 104. In one embodiment,the learning component 510 is configured to update a model or algorithmused by the modeling component 504. For example, the learning component510 may generate or modify at least a portion of the model based onmachine learning for known locations of obstacles detected by one ormore sensors and based on causal or deterministic constraints for knownphysical laws relating to the sensors or driving of the vehicle. In oneembodiment, the learning component 510 may modify a parameter, function,or structure of the model based on machine learning for one or moredriving events experienced by the vehicle. For example, sensors may beused to gather sensor data during a driving event. During the drivingevent, inferences determined by the inference component 506 may be madeand used for driving the vehicle. After the event has ended, additionalinformation may be available, which was not necessarily available duringthe event. For example, more accurate calculations regarding what reallyhappened during an event may be performed or the results of an event maybe detected. Using this more accurate, additional information, or theresult, the learning component 510 may revise parameters or otheraspects of a model to improve operation of the model component 504 andinference component 506. Similarly, the learning component 510 mayupdate an algorithm used by the inference component 506 to infer detailsbased on sensor data and/or a graphical model.

Referring now to FIG. 6, a schematic flow chart diagram of a method 600for sensor fusion is illustrated. The method 600 may be performed by anautomated driving/assistance system or a sensor fusion component, suchas the automated driving/assistance system 102 of FIG. 1 or the sensorfusion component 104 of FIG. 1 or 5.

The method 600 begins as the sensor data component 502 obtains sensordata for a region near a vehicle from two or more sensors at 602. Forexample, the sensor data component 502 may obtain 602 the sensor datafrom a radar system 106, a LIDAR system 108, a camera system 110, a GPS112, an ultrasound system 114, a wheel encoder, or any other sensor thatis observing a region near a vehicle. The model component 504 calculatesvalues in a joint-probabilistic graphical model based on the sensor dataat 604. The graphical model may include nodes that correspond to randomvariables and edges indicating correlations between the nodes. In oneembodiment, one or more functions may describe how to calculate a valuefor a node based on sensor data and relationships with other nodes. Theinference component 506 detects and tracks obstacles near a vehiclebased on the sensor data and the model using a WISH algorithm at 606.For example, the inference component 506 may detect and track obstaclesusing hashing functions and weighted sums to estimate the detection,position, and/or velocity of objects near the vehicle.

Referring now to FIG. 7, a schematic flow chart diagram of a method 700for sensor fusion is illustrated. The method 700 may be performed by anautomated driving/assistance system or a sensor fusion component, suchas the automated driving/assistance system 102 of FIG. 1 or the sensorfusion component 104 of FIG. 1 or 5.

The method 700 begins as the sensor data component 502 receives sensordata from two or more sensors for a region near a vehicle at 702. Themodel component 504 stores a joint-probabilistic graphic model thatmodels random variables and relationships between the random variablesbased on the two or more sensors at 704. In one embodiment, the modellogically divides the region near the vehicle into a grid and aplurality of nodes of the graphical model represent a binary occupancyof respective cells of the grid. The inference component 506 processesthe sensor data and the model using a dimensionality reduction algorithmto detect and track physical objects at 706. For example, the inferencecomponent 506 may process the sensor data and model using a WISHalgorithm.

EXAMPLES

The following examples pertain to further embodiments.

Example 1 is a system that includes two or more sensors configured toobtain sensor data. The system also includes a model component and aninference component. The model component is configured to calculatevalues in a joint-probabilistic graphical model based on the sensordata. The graphical model includes nodes corresponding to randomvariables and edges indicating correlations between the nodes. Theinference component is configured to detect and track obstacles near avehicle based on the sensor data and the model using a WISH algorithm.

In Example 2, the inference component of Example 1 further determines aconfidence bounds for one or more of a speed, a position, and a velocityof an obstacle relative to the vehicle.

In Example 3, one or more of the nodes in any of Examples 1-2 correspondto an occupancy of a cell in a grid describing a region observed by thetwo or more sensors and the inference component determines one or moreof a probability that the cell is occupied by an object and a velocityof the object occupying the cell.

In Example 4, one or more of the nodes in any of Examples 1-3 correspondto one or more continuous variables for an object detected by the two ormore sensors. The continuous variables include one or more ofreflectivity and spectral signature of an obstacle.

In Example 5, one or more of the nodes in any of Examples 1-4 correspondto one or more categorical variables for an obstacle. The categoricalvariables include one or more of an object type and a material type.

In Example 6, the model in any of Examples 1-5 includes a function forcalculating a value at a node based on the sensor data. The functionincludes a predetermined machine learning parameter to calculate thevalue.

In Example 7, the model in any of Examples 1-6 includes jointprobabilistic functions for one or more edges describing how nodescorresponding to the edges are mathematically related.

In Example 8, the system of any of Examples 1-7 further includes astorage component to store the joint-probabilistic graphical model.

In Example 9, the system of any of Examples 1-8 further include alearning component configured to generate or modify at least a portionof the model based on machine learning for known locations of obstaclesdetected by one or more sensors and based on causal or deterministicconstraints for known physical laws relating to the sensors or drivingof the vehicle.

In Example 10, the two or more sensors in any of Examples 1-9 includeone or more of a LIDAR system, a camera system, an ultrasound system, apositioning system, and wheel encoders.

Example 11 is a method implemented by a computing device. The methodincludes receiving sensor data from two or more sensors for a regionnear a vehicle. The method includes storing a joint-probabilisticgraphic model that models random variables and relationships between therandom variables based on the two or more sensors. The model logicallydivides the region near the vehicle into a grid and a plurality of nodesof the graphical model represent a binary occupancy of respective cellsof the grid. The method includes processing the sensor data and themodel using a dimensionality reduction algorithm to detect and trackphysical objects.

In Example 12, the dimensionality reduction algorithm of Example 11includes a WISH algorithm.

In Example 13, the method of any of Examples 11-12 further includesproviding a location or a velocity of the physical objects to anautomated driving system or automated assistance system.

In Example 14, the method of any of Examples 11-13 further includesselecting a driving path, using the processor, based on the detectionand tracking of the physical objects.

In Example 15, the model of any of Examples 11-14 includes parametersdetermined based on one or more of deterministic constraints based onphysical laws for the vehicle and probabilistic values for confidencelevels for detecting and tracking the physical objects.

In Example 16, the method of any of Examples 11-15 further includesmodifying at least a portion of the model based on machine learning forone or more driving events experienced by the vehicle.

Example 17 is computer readable storage media storing instructions that,when executed by one or more processors, cause the processors to:receive sensor data for a region near a vehicle from two or moresensors; store a joint-probabilistic graphic model that models randomvariables and relationships between the random variables based on thetwo or more sensors; and process the sensor data based on thejoint-probabilistic graphic model to detect and track one or morephysical objects near the vehicle. Processing the sensor data involvesprocessing based on the joint-probabilistic graphic model using a WISHalgorithm.

In Example 18, the computer readable storage media of Example 17 furtherincludes instructions that cause the processor to provide a location ora velocity of the one or more physical objects to an automated drivingsystem or automated assistance system of the vehicle.

In Example 19, processing the sensor data in any of Examples 17-18includes processing the sensor data based on one or more virtualenvironments indicated by the model.

In Example 20, the model of any of Examples 17-19 logically divides theregion near the vehicle into a grid, and a plurality of nodes of thegraphical model represent an occupancy of respective cells of the grid.Processing the sensor data based on the joint-probabilistic graphicmodel includes processing to determine probability of occupancy in eachcell with a confidence bounds.

Example 21 is a system or apparatus that includes means for implementingor realizing any of Examples 1-20.

It should be noted that the sensor embodiments discussed above maycomprise computer hardware, software, firmware, or any combinationthereof to perform at least a portion of their functions. For example, asensor may include computer code configured to be executed in one ormore processors, and may include hardware logic/electrical circuitrycontrolled by the computer code. These example devices are providedherein purposes of illustration, and are not intended to be limiting.Embodiments of the present disclosure may be implemented in furthertypes of devices, as would be known to persons skilled in the relevantart(s).

Embodiments of the disclosure have been directed to computer programproducts comprising such logic (e.g., in the form of software) stored onany computer useable medium. Such software, when executed in one or moredata processing devices, causes a device to operate as described herein.

While various embodiments of the present disclosure have been describedabove, it should be understood that they have been presented by way ofexample only, and not limitation. It will be apparent to persons skilledin the relevant art that various changes in form and detail can be madetherein without departing from the spirit and scope of the disclosure.Thus, the breadth and scope of the present disclosure should not belimited by any of the above-described exemplary embodiments, but shouldbe defined only in accordance with the following claims and theirequivalents. The foregoing description has been presented for thepurposes of illustration and description. It is not intended to beexhaustive or to limit the disclosure to the precise form disclosed.Many modifications and variations are possible in light of the aboveteaching. Further, it should be noted that any or all of theaforementioned alternate implementations may be used in any combinationdesired to form additional hybrid implementations of the disclosure.

Further, although specific implementations of the disclosure have beendescribed and illustrated, the disclosure is not to be limited to thespecific forms or arrangements of parts so described and illustrated.The scope of the disclosure is to be defined by the claims appendedhereto, any future claims submitted here and in different applications,and their equivalents.

What is claimed is:
 1. A system comprising: two or more sensorsconfigured to obtain sensor data; a model component configured tocalculate values in a joint-probabilistic graphical model based on thesensor data, wherein the graphical model comprises nodes correspondingto random variables and edges indicating correlations between the nodes;an inference component configured to detect and track obstacles near avehicle based on the sensor data and the model using aweighted-integrals-and-sums-by-hashing (WISH) algorithm; and anautomated driving system or an automated assistance system of thevehicle configured to execute a driving path for the vehicle based onthe detected and tracked obstacles near the vehicle.
 2. The system ofclaim 1, wherein the inference component determines a confidence boundsfor one or more of a speed, a position, and a velocity of an obstaclerelative to the vehicle.
 3. The system of claim 1, wherein one or moreof the nodes correspond to an occupancy of a cell in a grid describing aregion observed by the two or more sensors; and wherein the inferencecomponent determines one or more of a probability that the cell isoccupied by an object, and a velocity of the object occupying the cell.4. The system of claim 1, wherein one or more of the nodes correspond toone or more continuous variables for an object detected by the two ormore sensors, wherein the continuous variables comprise one or more ofreflectivity and spectral signature of an obstacle.
 5. The system ofclaim 1, wherein one or more of the nodes correspond to one or morecategorical variables for an obstacle, wherein the categorical variablescomprise one or more of an object type and a material type.
 6. Thesystem of claim 1, wherein the model comprises a function forcalculating a value at a node based on the sensor data, wherein thefunction comprises a predetermined machine learning parameter tocalculate the value.
 7. The system of claim 1, wherein the modelincludes joint probabilistic functions for one or more edges describinghow nodes corresponding to the edges are mathematically related.
 8. Thesystem of claim 1, further comprising a storage component to store thejoint-probabilistic graphical model.
 9. The system of claim 1, furthercomprising a learning component configured to generate or modify atleast a portion of the model based on machine learning for knownlocations of obstacles detected by one or more sensors and based oncausal or deterministic constraints for known physical laws relating tothe sensors or driving of the vehicle.
 10. The system of claim 1,wherein the two or more sensors comprises one or more of a lightdetection and ranging (LIDAR) system, a camera system, an ultrasoundsystem, a positioning system, and wheel encoders.
 11. A methodimplemented by a computing device, the method comprising: receivingsensor data from two or more sensors for a region near a vehicle;storing a joint-probabilistic graphic model that models random variablesand relationships between the random variables based on the two or moresensors, wherein the model logically divides the region near the vehicleinto a grid, and wherein a plurality of nodes of the graphical modelrepresent a binary occupancy of respective cells of the grid; processingthe sensor data and the model using a dimensionality reduction algorithmto detect and track physical objects; selecting a driving path based onthe detection and tracking of the physical objects; and providing thedriving path to an automated driving system or automated assistancesystem of the vehicle for executing the driving path by the vehicle. 12.The method of claim 11, wherein the dimensionality reduction algorithmcomprises a weighted-integrals-and-sums-by-hashing (WISH) algorithm. 13.The method of claim 11, further comprising providing a location or avelocity of the physical objects to the automated driving system or theautomated assistance system.
 14. The method of claim 11, furthercomprising determining a confidence bounds for one or more of a speed, aposition, and a velocity of a physical object relative to the vehicle.15. The method of claim 11, wherein the model comprises parameters thatare determined based on one or more of deterministic constraints basedon physical laws for the vehicle and probabilistic values for confidencelevels for detecting and tracking the physical objects.
 16. The methodof claim 11, further comprising modifying at least a portion of themodel based on machine learning for one or more driving eventsexperienced by the vehicle.
 17. Non-transitory computer readable storagemedia storing instructions that, when executed by one or moreprocessors, cause the one or more processors to: receive sensor data fora region near a vehicle from two or more sensors; store ajoint-probabilistic graphic model that models random variables andrelationships between the random variables based on the two or moresensors; process the sensor data based on the joint-probabilisticgraphic model to detect and track one or more physical objects near thevehicle, wherein processing the sensor data comprises processing basedon the joint-probabilistic graphic model using aweighted-integrals-and-sums-by-hashing (WISH) algorithm; select adriving path based on the detection and tracking of the one or morephysical objects near the vehicle; and providing the driving path to anautomated driving system or automated assistance system of the vehiclefor executing the driving path by the vehicle.
 18. The non-transitorycomputer readable storage media of claim 17, further comprisinginstructions configured to cause the one or more processors to provide alocation or a velocity of the one or more physical objects to theautomated driving system or the automated assistance system of thevehicle.
 19. The non-transitory computer readable storage media of claim17, wherein processing the sensor data comprises processing the sensordata based on one or more virtual environments indicated by the model.20. The non-transitory computer readable storage media of claim 17,wherein the model logically divides the region near the vehicle into agrid, and wherein a plurality of nodes of the graphical model representan occupancy of respective cells of the grid, wherein processing thesensor data based on the joint-probabilistic graphic model comprisesprocessing to determine probability of occupancy in each cell with aconfidence bounds.